Distinguish primary vs. secondary data sources and when to use each
Design effective questionnaires and survey instruments
Select appropriate data collection methods for your research question
Apply observation techniques to a real-world scenario
Artifact Output: What You Leave With
Today’s Deliverable
By end of class, you’ll have completed the Observation Design Exercise (jay-walking study):
Observation method selection with justification
Behavior categories defined
Reliability strategy outlined
This directly prepares you for Doc 0.2 (Methodology).
Introducing Data before its Collection
Types of Research Data Collection (Source)
Primary Data Source
1st hand information
not changed by any individual
not published yet
directly collected by authors
Secondary Source
published data
what literature review is based on
reviewed by authors(you)
Examples of data collection methods by category
Primary Data Collection Examples
Questionnaires
Interviews
Focus Group Interviews
Observation
participant/non-participant
aware/non-aware
Survey
Statistical Methods
Experimental Methods
Secondary Data Collection Examples
Published Papers/Sources
Databases
Books
General websites
Unpublished personal records
Census data/population statistics
Primary Data Collection Methods: Designing Questionnaires
a set of questions and secure answers from respondents
often analyzed by statistical methods
consistency in questionnaires make cross-sectional analysis easy
Types of questions designed to measure variables in survey:
Close-end questions
Two-option aka dichotomous scales
More than two options: Nominal-polychromous
Ordinal-Polytomous scale
Continuous or bounded types
Open-end questions
sentence completion
open-ended questions with free text responses
Polytomous Variables, aka Options for Multiple Choice Questions
Statistical term that refers to a categorical variable with more than two possible categories or levels
Common in: Survey research ,healthcare, market research and education
Characteristics:
Categorical data: limited/distinct values or categories that are mutually exclusive
Always more than two options(binary)
Ordered and unordered (natural hierarchy)
Statistical analyses: e.g. logistic regression, cluster analysis for patterns and trends
Convertable to binary variables
Purpose:
Understanding complex phenomena: patterns and trends in the data collected
Provide insights to better categorization of data
Statistical analyses: Relationships between variables: e.g. logistic regression to predict probability
Market segmentation: e.g. customers to preference, behavior, demographic groups
Polytomous Variable (Continued)
When to use Polytomous Variables
Measuring attitudes or perceptions: for more nuanced perceptions
Putting definitions on categorizzing data: greater variability
Analyzing relationships between variables: job satisfication & job performance
Need to capture complex phenomena
Limitations
Power of explanation limited by available options
e.g. satisfactory survey & number of enrolled students, what about change?
subjectivity from questionnaire design and interpretation of results
response bias
small sample sizes
calculating the appropriate size of population needed
amount of respondents needed to reach statistical significance (reject null hypothesis, not covered by current class, leverage sample size calculator online)
Primary Data Collection: Designing Questionnaires
Face-to-face, paper-and-pencil or remote, make sure your data collected is: - well-organized and - easily accessible for analysis.
General rules for constructing a questionnaire:
Dos:
questions should be short and simple
provide clear navigation to avoid difficulty in reading and motivate answering
use positive sentences
add open-answer possibility after provideing listed answers
improve reliability by selecting appropriate words
explain importance of the questionnaire
order your questions to solicit the right answers (sensitive to follow concrete/innocuous ones)
Do-nots:
use more than one question (double-barreled) in one item
make assumptions for the respondents
lead the respondent to answers with clues, suggestions and hints
Steps involved in designing a questionnaire
Primary Data Collection: Interviews
Face-to-face and remote (telephone/zoom) interviews and merit/demerits (Kabir, 2016).
Good for complex or sensitive concepts and need detailed and high-status information (Frechtling, 2020).
Types of interviews by structure:
structured interviews: standardized questions that are pre-prepared
semi-structured interviews: conducted based on guide but goes beyond list of questions
Public space, no recording of faces, aggregate data only
This type of study informs urban design guidelines and policy.
Let’s Practice: Designing an Observation Study
Several years ago, a group of students at University of Central Arkansas conducted a study in which they observed the rate at which cars failed to stop at a campus stop sign and recorded whether the car had a student parking decal or a faculty/staff parking decal. This is obviously not fitting for Hong Kong context. Let’s perhaps picture a study of the rate of jay-walking at a traffic light instead - and record whether the pedestrian who crossed is a student/staff/tourist/local resident. Use what we have covered today to answer questions 1-7:
Which method of observation would be best? Justify your answer. Hint: back to participant/direct observations.
How would you schedule observations?
Define the categories of behavior that you would observe
Describe how you would optimize and measure the reliability of observations, including the use of independent observers and calculation of interobserver agreement.
Describe how you could use equipment for observation rather than human observers, what are the advantages and disadvantages?
Describe how you might use public records to answer the same research question. What might be some limitations of this approach
Describe how you might use a survey method to answer the same research question. What might be some limitations of this approach?
References
Frechtling, J. (2002). An overview of quantitative and qualitative data collection methods The 2002 user-friendly handbook for project evaluation (pp. 43-62).
Hox, J. J., & Boeije, H. R. (2005). Data collection, primary versus secondary Encyclopedia of social Measurement (Vol. 1): Elsevier.
Data collection challenges (2005).
Kabir, S. M. S. (2016). Methods Of Data Collection Basic Guidelines for Research: An Introductory Approach for All Disciplines (first ed., pp. 201-275).
Olsen, W. (2012). Data collecti on: Key debates and methods in social research (Vol. 1): Sage.
Pandey, P., & Pandey, M. M. (2015). Research Methodology: Tools and Techniques (Vol. 1). Romania: Bridge Center.
Rimando, M., Brace, A. M., Namageyo-Funa, A., Parr, T. L., Sealy, D.-A., Davis, T. L., . . . Christiana, R.W. (2015). Data collection challenges and recommendations for early career researchers. The Qualitative Report, 20 (12), 2025-2036.
Taherdoost, H. (2016a). How to design and create an effective survey/questionnaire; A step by step guide. International Journal of Academic Research in Management (IJARM), 5 (4), 37-41.
Taherdoost, H. (2016b). Measurement and scaling techniques in research methodology; survey/questionnaire development. International Journal of Academic Research in Management (IJARM),6 (1), 1-5.
Taherdoost, H. (2016c). Sampling methods in research methodology; how to choose a sampling technique for research. International Journal of Academic Research in Management (IJARM), 5 (2), 18-27.
Taherdoost, H. (2019). What is the best response scale for survey and questionnaire design; review of different lengths of rating scale/attitude scale/Likert scale. International Journal of Academic Research in Management (IJARM), 8 (1), 1-10.
Taherdoost, H. (2021). Handbook on Research Skills: The Essential Step-By-Step; Guide on How to Do a Research Project (Kindle ed.): Amazon.
Learning Objectives
By the end of this session, you will be able to:
Prepare for informational interviews through professional research
Craft effective outreach messages for professional networking
Apply thematic analysis to qualitative data (codes → themes)
Articulate the “so what?” of your findings
Reflecting on Literature Review Submissions
Common Patterns in Doc 0.1 Submissions
What typically works well:
Clear articulation of research focus
Adequate number of sources
Common areas for improvement:
Formatting doesn’t match discipline conventions
Synthesis missing — reads as summary, not analysis
Theoretical framework underdeveloped
Conclusions included (lit reviews don’t conclude — they set up your contribution)
Quick flash back on Document 1 submission
Research-paper like
~80% of work needed to draft a research paper as it contains:
familiarize with the professional’s role/projects/contributions
understanding the context and significance of their work
Contextual Understanding
Know the Industry: major trends, challenges and opportunities
Company Culture: culture and values of where they work/associate with for talking points
Know Your Client (KYC) Cont’d
Preparing Questions: Structured/Semi-structured
Informed Questions: Informed and Open-Ended, reflecting their research and curiosity about the professional
Personalized Inquiry: avoid overly general or easily Google-able questions
Object-Oriented: what do you want to achieve through the interviews
get first-hand critique on the discipline
know what day-to-day is like at the particular industry
Set an agenda: what will happen if you don’t have one?
Activity: Research a Professional (10 min)
Do background research on a potential interviewee you might be interested in reaching out to
you should cover at least the following:
their professional role
list out companies they worked for
their career highlight/work most proud of
identify interviews/podcasts/presentation where they gave public opinion and based on those
come up with two questions that you think is relevant to ask
Meet Your Client
Showing Engagement:
Active Engagement:
active listening,
showing interest through body language (in person or on video calls), and
asking follow-up questions based on the discussion.
Note-taking:
taking brief notes during the conversation (with permission) to capture key points and advice,
demonstrates engagement and respect for the professional’s insights.
Respect and Professionalism:
Time Management:
be mindful of the agreed-upon time for the conversation and
to avoid overextending unless the professional indicates they’re willing to continue.
Confidentiality:
the need to respect any confidential or sensitive information shared during the conversation.
Meet Your Client (Continued)
Post-Interaction Reflection:
Reflective Practice:
Review and Reflect:
review their notes and reflect on the conversation soon after,
identifying key learnings and any follow-up actions they might take.
Feedback Loop:
consider what went well and what could be improved for future interactions
mindset of continuous improvement.
Mock Interview (If we have time ~ 15min)
one-on-one practice: professional vs. student interviewer
work together and determine who the professional will shadow (a real person)
KYC on industry/persona individually
student interviewer to come up with 3 questions as structured interview and one follow-up during the conversation
student/professional to swap (if we have time)
Seminar Time: Back to Report Submission with Doc 0.1
Activities
Vote to review one literature review that was submitted (anonymized on mentimeter).
Selected literature reviewed by peers with the following criteria:
Objective: Does the literature review sufficiently present its objective?
Landscape: Do you think the literature review presented a clear theoretical framework of what are the relevant studies and state-of-the art research landscape?
Caveat/Importance: Does the literature review present any clear indication that there is room for investigation of any existing caveats?
Citations: Do you think the number of citations included is enough/adequate?
Improvements: Anything that the author can do to strengthen the literature review?
Challenge: On the topics that were discussed in the literature review, can you quickly leverage google scholar to find additional ones that investigates similar things?
We will review the remainder three of the five documents (so long as time permits) for this week.
Addendum: Checklist for successful literature review
1. Clarity and Coherence:
Clear Objective: Define the specific goals and scope of the literature review.
Logical Structure: Organize content in a logical manner, facilitating easy navigation and understanding.
Language and Terminology: Use clear, concise language appropriate for an interdisciplinary audience, avoiding unnecessary jargon.
2. Problem Statement and Importance:
Well-defined Issue: Clearly articulate the problem or research question the literature review addresses.
Significance: Explain the importance and relevance of the problem within the context of design and computer science.
Gap Identification: Highlight gaps or shortcomings in existing research that the literature review aims to address.
Addendum: Checklist for successful literature review (Cont’d)
3. Theoretical Framework:
Conceptual Underpinnings: Present the theories or models that underlie the research area.
Framework Integration: Demonstrate how the theoretical framework informs the literature review’s approach and analysis.
Interdisciplinary Relevance: Ensure the framework is relevant and accessible to both design and computer science perspectives.
4. Comprehensive Coverage:
Breadth and Depth: Cover a wide range of sources while diving deep into critical studies.
Diverse Sources: Include academic journals, conference papers, books, and reputable online resources relevant to both fields.
Timeliness: Ensure the inclusion of both foundational texts and recent research to reflect the current state of knowledge.
5. Critical Analysis:
Comparative Analysis: Compare and contrast different studies, highlighting similarities and differences.
Methodological Evaluation: Assess the methodologies used in key studies for their strengths and limitations.
Theoretical Critique: Critically evaluate the theories discussed in the literature for their applicability and limitations in the interdisciplinary context.
Addendum: Checklist for successful literature review (Cont’d)
6. Synthesis:
Thematic Organization: Synthesize literature thematically rather than summarizing each source individually.
Insight Generation: Derive new insights or perspectives from the synthesis of the reviewed literature.
Interdisciplinary Integration: Fuse insights from design and computer science to create a cohesive understanding.
7. Relevance to Research Question:
Alignment: Ensure all reviewed literature contributes to answering the research question or addressing the problem statement.
Application: Discuss how findings from the literature review apply to the specific intersection of design and computer science.
8. Source Evaluation:
Credibility Assessment: Evaluate the credibility and reliability of each source.
Bias and Perspective: Acknowledge potential biases in the literature and strive for a balanced perspective.
Addendum: Checklist for successful literature review (Cont’d)
9. Conclusions and Implications:
Summary of Findings: Concisely summarize key findings and their implications for the research area.
Future Research Directions: Identify areas where further research is needed, especially at the intersection of design and computer science.
10. Documentation and Referencing:
Consistent Formatting: Adhere to a consistent citation style appropriate for the interdisciplinary audience.
Accurate Citations: Ensure all sources are accurately cited within the text and in the reference list.
11. Reflection on Interdisciplinarity:
Integration Challenges: Discuss any challenges encountered in integrating design and computer science literature and how they were addressed.
Value of Interdisciplinary Approach: Reflect on how the interdisciplinary approach enriches the understanding of the topic.
Revisiting Report Submission
Document 0: Research Statement (Extended Abstract)
Document 0.1: Literature Review
Document 0.2: Methodology & Data Needed/Collected
Document 1: Research Proposal (Paper-like)
Check Moodle for current due dates.
Research Process in Flow Chart
Checklist for your final research proposal/paper submission
Elements that you’ve finalized:
Research Problem Definition (Doc 0):
Articulated innovative challenge or aspect addressed.
Contextualized within industry/societal needs.
Relevant Concepts and Theories (Doc 0.1):
Reviewed key theories underpinning the innovation area.
Included interdisciplinary approaches.
Previous Research Findings (Doc 0.1):
Highlighted past innovations and research.
Noted successes and gaps for building upon/addressing.
Ready to be finalized:
Hypothesis Formulation (Doc 0.2):
Developed clear, testable hypotheses predicting innovation outcomes.
Research Design Formulation (Doc 0.2):
Outlined appropriate research methodologies/technologies to be used.
Discussed feasibility and potential impact of proposed research design.
Research Progress Check
Insofar, Your Design Should Provide:
Evidence for Hypothesis:
Detailed data collection/analysis plan for hypothesis testing.
Consideration on data sources/appropriate technologies deployed in their collection.
Significance:
Connected significance to societal/industry/academic trends.
Highlighted contribution to knowledge/innovation advancement.
Implementation and Scalability:
Discuss real-world implementation pathways.
Address potential scalability barriers.
Ethical Considerations and Sustainability:
Outline ethical considerations of the innovation.
Address sustainability of the proposed solution.
Final Step in Finalizing Research
As Finalizing the Proposal (Doc 1):
Integration and Synthesis:
Ensured alignment of all proposal components with innovation/research goals.
Synthesis Activity (25 min) — Working with the Synthesis Canvas
Research → Decision (20 min) — Product Track framework
Logistics (5 min) — Interview project reminder
Descriptive vs. Analytical Writing
Welcome to today’s session on distinguishing between descriptive and analytical writing. Let’s enhance our academic writing skills together!
Activity: Identify the Writing Style
We’ll go through several examples. For each, determine if it’s descriptive or analytical. After your response, we’ll reveal the correct answer and discuss.
Example 1: Question
“User engagement metrics have decreased by 20% over the past three months.”
Question:
Is this statement descriptive or analytical?
Example 1: Answer
Answer: Descriptive
Explanation:
The sentence reports a fact (a 20% drop) without delving into any reasons or implications behind the metric change.
Example 2: Question
“Maslow’s hierarchy of needs is a psychological theory that arranges human needs in a pyramid, with physiological needs at the base and self-actualization at the top.”
Question:
Is this statement descriptive or analytical?
Example 2: Answer
Answer: Descriptive
Explanation:
While it explains the pyramid structure, it does not analyze or critique the theory’s applicability or limitations across different cultures.
Example 3: Question
“The experiment demonstrated that plants exposed to sunlight grew faster than those kept in the shade.”
Question:
Is this statement descriptive or analytical?
Example 3: Answer
Answer: Descriptive
Explanation:
The statement provides an observation (faster growth with sunlight) without discussing the process or significance of the finding.
Example 4: Question
“In Shakespeare’s ‘Macbeth,’ Lady Macbeth’s manipulation of her husband’s actions serves as a commentary on the corrupting power of unchecked ambition and challenges traditional gender roles in Shakespearean society.”
Question:
Is this statement descriptive or analytical?
Example 4: Answer
Answer: Analytical
Explanation:
It not only identifies Lady Macbeth’s influence but also interprets her actions as a critique of societal norms, thus offering analytical insight.
Example 5: Question
“Survey results show that 60% of respondents prefer online shopping over in-store shopping.”
Question:
Is this statement descriptive or analytical?
Example 5: Answer
Answer: Descriptive
Explanation:
It presents a statistic without discussing why users might prefer online shopping or its implications for consumer behavior.
Example 6: Question
“While the interface layout includes a navigation bar, a content area, and a footer, this design might impede user engagement if it fails to prioritize the most critical user tasks, suggesting a need for a more dynamic, behavior-driven layout.”
Question:
Is this statement descriptive or analytical?
Example 6: Answer
Answer: Analytical
Explanation:
The statement goes beyond description by assessing the potential drawbacks of the layout and suggesting improvements based on user behavior.
Key Takeaways
Descriptive vs. Analytical:
Descriptive writing states facts or observations.
Analytical writing explains the significance, reasoning, and implications behind those facts.
Depth of Analysis:
Effective academic writing connects evidence with critical interpretation.
Always ask: “Why is this important?” and “What does this mean for the overall argument?”
Practical Strategies for Improvement:
Peer Review: Leverage feedback from classmates to identify areas lacking analysis.
Reverse Outlining: Break down your text to pinpoint descriptive versus analytical sections.
Continuous Revision:
Use these insights to refine your drafts, deepening your analysis and improving clarity.
Enhancing Your Analytical Writing
Objective:
- Develop your analytical writing skills by examining and revising your literature review or methodology sections.
- Engage in collaborative activities to practice critical analysis.
Today’s Activities:
1. Peer Review: In pairs, analyze each other’s writing using guided questions.
2. Reverse Outlining: Break down a text (yours or a provided sample) to identify strengths and gaps.
Stage 1: Peer Review Activity
Time: 25 minutes
Instructions:
Pair Up: Find a partner in the room.
Exchange Materials: Share your literature review or methodology section (or use the provided sample).
Guided Review: Use these questions as you read your partner’s work:
Thesis & Argument:
Is the thesis clear and arguable?
Does the argument show critical evaluation rather than just stating facts?
Use of Evidence:
Is evidence well integrated and analyzed?
Does the writer explain the significance of each piece of evidence?
Structure & Flow:
Are the ideas presented logically?
Do transitions connect the analysis throughout the paper?
Goal:
- Provide constructive feedback focused on deepening the analysis.
Stage 2: Reverse Outlining Exercise
Time: 25 minutes
Instructions:
Choose Your Text: Pick a section from the previously worked-upon example you looked at (4-5 paragraphs max).
Number Each Paragraph: Number the paragraphs in the margin for clarity.
Summarize Each Paragraph: In one sentence, capture the main idea or argument of each paragraph.
Evaluate the Outline:
Does each summary contribute to your overall argument?
Identify any parts that are merely descriptive rather than analytical.
Plan Revisions: Mark paragraphs that need more critical insight or better integration of evidence.
Goal:
- Reveal the structure and flow of your argument and present to each other. - Identify areas where you can add deeper analysis or clarity.
Reflection & Discussion
Discussion Questions:
- What new insights did you gain about your own writing from these activities?
- How can the feedback from your peer review inform your revisions?
- What specific changes will you make to improve the analytical depth of your paper?
Next Steps:
- Revise your draft based on the insights and feedback received. - Schedule a follow-up session if you need additional support on integrating analytical writing techniques.
Double-check your submission to make sure that it can be considered a followable instruction manual by the domain experts.
Synthesis Activity
Using the Synthesis Canvas (25 min)
The challenge: You have findings from multiple sources. How do you bring them together?
Fill in Section 1 of the canvas: - What qualitative data? (interviews, observations, open-ended responses) - What quantitative data? (survey scores, analytics, measurements) - How much of each?
Activity: Qualitative Synthesis (10 min)
Step 2: Pull Key Quotes
From your interviews/observations, identify 3-5 powerful quotes that capture key insights.
Step 3: Group Into Themes
Cluster related quotes. Name each theme. Note how many participants expressed it.
Theme
Codes Included
# of Participants
Theme 1: _________
_________
/
Theme 2: _________
_________
/
Activity: Triangulation Check (7 min)
Step 4: Do Qual and Quant Agree?
For each finding, check:
Finding
Qual Evidence
Quant Evidence
Agreement?
_________
_________
_________
Yes / No / Partial
If they disagree: That’s interesting data. Write down why they might differ.
Activity: The “So What?” (5 min)
Step 5: Why Should Anyone Care?
For your top 2-3 findings, write one sentence explaining the implication:
Finding
So What?
_________
This means… / This suggests… / Designers should consider…
If you can’t answer “so what?” — the finding isn’t ready for Doc 1.
Architecture/Urban Example: Synthesizing Mixed Methods Data
Research question: Does greenery in Hong Kong’s elevated walkways affect pedestrian stress levels?
Data Type
Source
Key Finding
Qualitative
10 walking interviews
“The plants make it feel less like a tunnel” (7/10)
Quantitative
Heart rate variability (N=45)
12% lower stress markers in green sections vs. bare sections
Observation
Pedestrian counts
23% more people paused/lingered in green sections
Triangulation: All three methods converge — greenery correlates with lower stress and changed behavior.
So what? Urban designers should prioritize vegetation in elevated walkway design; the effect is measurable, not just aesthetic preference.
Research → Decision Framework
Product Track Focus
This section is specifically for students on the Product Track who are building design case studies rather than academic papers. If you’re on the Research Track, this framework shows how industry applies research — useful context even if your output is a traditional paper.
The Product Builder’s Challenge
You’ve done your research. You found the behavioral science. Now what?
Academic output: “Our findings suggest that social comparison mediates the relationship between like-checking frequency and anxiety (β = 0.38, p < .001).”
Product question: “Should we hide like counts? Will it actually help? How will we know?”
This section bridges that gap.
The Translation Problem
What Research Gives You
What Product Needs
Correlations & themes
Causal hypotheses
General principles
Specific feature specs
“This is happening”
“If we do X, Y will happen”
Academic language
Stakeholder-ready framing
Your job: Turn “what we know” into “what we should build and why we expect it to work.”
The Research → Decision Canvas
Component
Question
Your Answer
Scientific Foundation
What does behavioral science say about this problem?
_____________
Product Hypothesis
If we build [feature], then [outcome] because [mechanism]
_____________
Expected Effect
How big a change do we expect? What’s the baseline?
_____________
Success Metric
How will we measure if it worked?
_____________
Risk if Wrong
What happens if our hypothesis is wrong?
_____________
Validation Plan
How will we test before full launch?
_____________
Worked Example: Instagram Hidden Likes
Research finding: Social comparison theory (Festinger, 1954) + your qualitative data showing 9/12 participants check likes to compare with friends.
Component
Answer
Scientific Foundation
Social comparison drives anxiety; visible metrics enable comparison; removing visibility should reduce comparison triggers
Product Hypothesis
If we hide like counts from viewers, then comparison-driven anxiety will decrease, because users can’t compare their posts to others’
Expected Effect
Baseline: 67% report checking to compare. Target: <40% report comparison motivation after hiding.
Success Metric
Self-reported comparison frequency (survey); anxiety scale (GAD-7); qualitative interviews
Risk if Wrong
Users might feel less validated, reducing posting motivation; engagement metrics might drop
Validation Plan
A/B test with 10% of users for 4 weeks; exit survey + 10 interviews
Case Study: Duolingo’s Streak Feature
The science: Loss aversion (Kahneman & Tversky, 1979) — people are more motivated to avoid losing something than to gain something of equal value.
The translation:
Component
Duolingo’s Answer
Scientific Foundation
Loss aversion: losing a streak feels worse than gaining a day
Product Hypothesis
If we visualize consecutive-day streaks, users will practice daily to avoid “losing” their streak
Expected Effect
Daily active users should increase; session frequency should stabilize
Success Metric
DAU/MAU ratio; 7-day retention; streak length distribution
Risk if Wrong
Users might feel punished by broken streaks and churn
Validation Plan
Introduced incrementally; added “streak freeze” as safety valve
Result: Streaks became Duolingo’s core retention mechanism.
Case Study: Spotify Wrapped
The science: Self-presentation theory (Goffman, 1959) — people curate identity through what they share; nostalgia increases emotional engagement.
The translation:
Component
Spotify’s Answer
Scientific Foundation
Sharing music = identity signaling; year-end reflection = nostalgia; social proof drives adoption
Product Hypothesis
If we create shareable, personalized year-in-review content, users will spread it organically, driving brand awareness and emotional connection
Expected Effect
Viral social sharing in December; increased premium conversions post-Wrapped
Success Metric
Social shares; hashtag volume; December premium signups vs. baseline
Risk if Wrong
Users might find it creepy or ignore it
Validation Plan
Started with simple stats; iterated based on what got shared most
Result: Wrapped became Spotify’s biggest annual marketing moment — built on behavioral science, not ad spend.
When Your Research Says “Don’t Build It”
Sometimes research reveals your hypothesis is wrong. That’s valuable.
Signals to reconsider:
Literature shows the mechanism doesn’t work the way you thought
Qualitative data shows users don’t actually have the problem you assumed
Comparable products tried it and failed (with documented reasons)
The effect size in existing research is too small to matter
What to do:
Document why you’re not building it (prevents revisiting the same dead end)
Look for adjacent opportunities the research revealed
Share with stakeholders — “We saved X months by researching first”
What Product Research Output Looks Like
Not a paper. A decision document.
Structure:
The Opportunity (1 paragraph): What problem are we solving? For whom?
What We Learned (1 page): Key findings from research — qual themes, quant patterns, literature insights
Our Hypothesis (1 paragraph): If we build X, then Y, because Z
Recommendation (1 paragraph): Build / Don’t build / Test first
Validation Plan (if building): How we’ll know if we’re right
Appendix: Raw data, quotes, methodology details
Length: 2-3 pages + appendix. Readable in 10 minutes.
The Screenshot-Worthy Slide
Research → Decision in One Sentence
“We found [scientific principle] in the literature and [pattern] in our data, which suggests that if we build [feature], users will [outcome] — and we’ll measure success by [metric].”
Example:
“We found that loss aversion (Kahneman & Tversky) predicts people work harder to protect what they have than to gain something new. Our interviews confirmed users hate ‘losing progress.’ This suggests that if we add a visible streak counter, users will practice daily to avoid breaking it — and we’ll measure success by 7-day retention rate.”
Artifact Output: What You Leave With
Today’s Deliverables
From Writing Workshop:
Reverse outline of your Doc 0.1 or Doc 0.2 draft
Peer feedback on descriptive vs. analytical writing
List of paragraphs that need deeper analysis
From Synthesis (Research Track):
Draft Synthesis Canvas with at least 2 themes + triangulation check
From Research → Decision (Product Track):
Completed Research to Decision Canvas with hypothesis and validation plan
These feed directly into Doc 1.
Logistics: Interview Project Reminder
Group/Individual Interview Project
Quick Reminder
The interview-based project (group or individual) gives you practice with:
Professional outreach and networking
Primary data collection through interviews
Synthesizing insights from first-hand sources
Choose group or individual based on your preferences. Check Moodle for current deadlines and submission guidelines.
A foundational course in research methodology for design practice. We explore how rigorous inquiry informs innovation—from framing questions to gathering evidence and communicating findings.
The Course’s Core Question
Can you distinguish incremental improvement from genuine innovation? Research methodology gives you the tools to tell the difference — and to pursue whichever path you choose deliberately.
Two Tracks, One Methodology:
Research Track
Product Track
Extended Abstract → Literature Review → Methodology → Research Paper
Product Research Brief → Landscape Intelligence → Research Playbook → Design Case Study
Choose based on your interests. Same rigor. Same skills. Different outputs.